Repository navigation
Honor CloudFetch download settings on kernel connections - #1060
Open
aakash-saravanan-db wants to merge 4 commits into
Open
aakash-saravanan-db wants to merge 4 commits into
aakash-saravanan-db wants to merge 4 commits into
Conversation
aakash-saravanan-db
temporarily deployed
to
azure-prod
November 14, 2026 04:11 — with
GitHub Actions
Inactive
aakash-saravanan-db
temporarily deployed
to
azure-prod
November 14, 2026 04:11 — with
GitHub Actions
Inactive
vuanhphung
approved these changes
Nov 16, 2026
vuanhphung
left a comment
Collaborator
There was a problem hiding this comment.
Have you evaluated whether 10 is a good default value for Kernel? I'm not sure if kernel implementation is similar to Thrift to have the same default value. The current Kernel default is 16 which will change after this PR
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
aakash-saravanan-db
force-pushed
the
aakash-saravanan_data/feat/cloudfetch-download-settings-20261114
branch
from
November 17, 2026 02:27
a8fb298 to
8a1c266
Compare
aakash-saravanan-db
temporarily deployed
to
azure-prod
November 17, 2026 02:27 — with
GitHub Actions
Inactive
aakash-saravanan-db
temporarily deployed
to
azure-prod
November 17, 2026 02:27 — with
GitHub Actions
Inactive
aakash-saravanan-db
temporarily deployed
to
azure-prod
November 17, 2026 02:27 — with
GitHub Actions
Inactive
aakash-saravanan-db
enabled auto-merge
November 17, 2026 05:54
aakash-saravanan-db
disabled auto-merge
November 17, 2026 08:05
aakash-saravanan-db
enabled auto-merge
November 17, 2026 08:05
aakash-saravanan-db
temporarily deployed
to
azure-prod
November 17, 2026 08:18 — with
GitHub Actions
Inactive
aakash-saravanan-db
disabled auto-merge
November 17, 2026 08:36
aakash-saravanan-db
enabled auto-merge
November 17, 2026 08:36
aakash-saravanan-db
disabled auto-merge
November 17, 2026 08:39
aakash-saravanan-db
enabled auto-merge
November 17, 2026 08:39
aakash-saravanan-db
disabled auto-merge
November 17, 2026 08:39
aakash-saravanan-db
enabled auto-merge
November 17, 2026 08:40
aakash-saravanan-db
disabled auto-merge
November 17, 2026 08:40
aakash-saravanan-db
enabled auto-merge
November 17, 2026 08:41
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature or gap
Python applications could set
max_download_threadson Thrift connections, but kernel connections ignored it.What this PR addresses
Forward the existing setting to the kernel download window, retaining defaults of 16 on the kernel and 10 on Thrift. Honor explicit settings, including 10, preserve raw chunk overrides, and document connection-time configuration, validation, the 256 ceiling and the distinction from a byte limit.
Dependencies
Existing native kernel APIs; no native dependency bump is required.
Tests and validation
354 focused tests pass, with stub-kernel and real-kernel suites run separately; live checks of the built wheel on Databricks SQL confirm window 16 and peak 16 downloads with the setting omitted, complete 5M-row results, reuse and close. With native kernel
16bc3c4b, equal warmups and five alternating measurements per setting give DBSQL submission-through-drain medians for 5M ordered rows: explicit window 16 at 24.573 s versus 10 at 24.434 s (-0.6%, parity); every result is complete and correct.